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[Document name] Specification 

[Title of the invention] Method and Apparatus For Reducing Processing 
Power Requirements of a Video Decoder 
[The claims] 

[Claim 1] An apparatus for reducing processing power requirements of a 
video decoder comprising: 

controlling means for controlling said processing power requirements 
of said decoder by using a predetermined throttling amount. 
[Claim 2] An apparatus according to claim 1, wherein determination of 
said determined throttling amount comprises the use of one or more of the 
following: 

one or more measures of the processing power required to decode 
one or more bitstreams; and 

one or more measures of said decoder's processing capabilities. 
[Claim 3] An apparatus according to claim 2, wherein said measures of 
the processing power required to decode a bitstream comprises of one or 
more of the following: 

an indication of the syntax elements in said bitstream; 

an indication of the type of processing required for said bitstream; 

an indication of the amount of processing required for said bitstream; 

a combination of the type and amount of processing required for said 
bitstream. 

[Claim 4] An apparatus according to claim 1, 2 or 3, wherein said 
controlling means for controlling said processing requirements comprises: 

means for reducing the processing power used for one or more 
decoder functions by limiting said decoder function in some way. 
[Claim 5] An apparatus according to claim 4, wherein said means for 
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limiting said decoder function comprises: 

means for turning off the use of one or more of the motion vectors 
normally used to retrieve a motion compensated reference. 
[Claim 6] An apparatus according to claim 4, wherein said means for 
limiting said decoder function comprises: 

means for selecting from a choice of motion vectors which refer to 
different reference pictures, one or more motion vectors to turn off; 

means for turning off the use of selected motion vectors normally used 
to retrieve a motion compensated reference. 

[Claim 7] An apparatus according to claim 6, wherein said means for 
selecting motion vectors comprises the following: 

means for selecting the motion vectors which are of the longest 
distance. 

[Claim 8] An apparatus according to claim 6, wherein said means for 
selecting motion vectors comprises the following: 

means for selecting the motion vectors which are least highly 
correlated with those motion vectors in surrounding regions of the picture 
region being decoded. 

[Claim 9] An apparatus according to claim 6, wherein said means for 
selecting motion vectors comprises the following: 

means for selecting the motion vectors which are of spatially longest 
distance. 

[Claim 10] An apparatus according to claim 6, wherein said means for 
selecting motion vectors comprises the following: 

means for selecting all motion vectors. 
[Claim 11] An apparatus according to claim 4, wherein said means for 
limiting said decoder function comprises: 
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means for modifying the value of one or more of the decoder's motion 
vectors. 

[Claim 12] An apparatus according to claim 1 1 , wherein said means for 
modifying the value comprises: 
5 means for limiting the precision of one or more of the decoder's motion 

vectors. 

[Claim 13] An apparatus according to claim 1 1 , wherein said means for 
modifying the value comprises: 

means for limiting the range of one or more of the decoder's motion 
10 vectors. 

[Claim 14] An apparatus according to claim 4, wherein said means for 
limiting said decoder function comprises: 

means for reducing the number of coefficients inverse quantized and 
inverse DCT transformed by selectively setting coefficients to alternate 
15 values. 

[Claim 15] An apparatus according to claim 14, wherein said alternate 
values comprises: 
zero. 

[Claim 16] An apparatus according to claim 4, wherein said means for 
20 limiting said decoder function comprises: 

means for eliminating the decoder processing of one or more of the 
color components. 

[Claim 17] An apparatus according to claim 4, wherein said means for 
limiting said decoder function comprises: 
25 means for reducing the amount of processing performed on decoded 

pictures preceding their display. 

[Claim 18] A method of reducing processing power requirements of a video 
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decoder comprising: 

controlling said processing power requirements of said decoder by 
using a predetermined throttling amount. 

[Claim 19] A method according to claim 18, wherein determination of said 
determined throttling amount comprises the use of one or more of the 
following: 

one or more measures of the processing power required to decode 
one or more bitstreams; and 

one or more measures of said decoders processing capabilities. 
[Claim 20] A method according to claim 19, wherein said measures of the 
processing power required to decode a bitstream comprises of one or more 
of the following: 

an indication of the syntax elements in said bitstream; 

an indication of the type of processing required for said bitstream; 

an indication of the amount of processing required for said bitstream; 

a combination of the type and amount of processing required for said 
bitstream. 

[Claim 21] A method according to claim 18, 19 or 20, wherein said 
controlling said processing requirements comprises: 

reducing the processing power used for one or more decoder functions 
by limiting said decoder function in some way. 

[Claim 22] A method according to claim 21, wherein said limiting said 
decoder function comprises: 

turning off the use of one or more of the motion vectors normally used 
to retrieve a motion compensated reference. 

[Claim 23] A method according to claim 21, wherein said limiting said 
decoder function comprises: 
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selecting from a choice of motion vectors which refer to different 
reference pictures, one or more motion vectors to turn off; 

turning off the use of selected motion vectors normally used to retrieve 
a motion compensated reference. 

[Claim 24] A method according to claim 23, wherein said selecting motion 
vectors comprises the following: 

selecting the motion vectors which are of the longest distance. 
[Claim 25] A method according to claim 23, wherein said selecting motion 
vectors comprises the following: 

selecting the motion vectors which are least highly correlated with 
those motion vectors in surrounding regions of the picture region being 
decoded. 

[Claim 26] A method according to claim 23, wherein said selecting motion 
vectors comprises the following: 

selecting the motion vectors which are of spatially longest distance. 
[Claim 27] A method according to claim 23, wherein said selecting motion 
vectors comprises the following: 

selecting all motion vectors. 
[Claim 28] A method according to claim 21 , wherein said limiting said 
decoder function comprises: 

modifying the value of one or more of the decoder's motion vectors. 
[Claim 29] A method according to claim 28, wherein said modifying the 
value comprises: 

limiting the precision of one or more of the decoder's motion vectors. 
[Claim 30] A method according to claim 28, wherein said modifying the 
value comprises: 

limiting the range of one or more of the decoder's motion vectors. 
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[Claim 31] A method according to claim 21, wherein said limiting said 
decoder function comprises: 

reducing the number of coefficients inverse quantized and inverse 
DCT transformed by selectively seltting coefficients to alternate values. 
[Claim 32] A method according to claim 31 , wherein said alternate values 
comprises: 

zero. 

[Claim 33] A method according to claim 21 , wherein said limiting said 
decoder function comprises: 

eliminating the decoder processing of one or more of the color 
components. 

[Claim 34] A method according to claim 21 , wherein said limiting said 
decoder function comprises: 

reducing the amount of processing performed on decoded pictures 
preceding their display. 
[Detailed description of the invention] 
[0001] 

[Application field in industry] 

This invention relates to the field of digital video decoders and the 
reduction of computational requirements of these decoders. 
[0002] 
[Prior art] 

Digital video has become widespread in the field of consumer 
electronics, due in large part to the emergence of digital video standards 
such as MPEG1 (see "CD1 1 172 - Coding of Moving Pictures and Associated 
Audio for Digital Storage Media at up to about 1.5 Mbps" by International 
Organisation for Standardisation, ISO MPEG Document, ISO- 
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IEC/JTC1/SC2/WG1 1 , 1994), MPEG2 (see "IS13818 - Generic coding of 
Moving Pictures and Associated Audio" by International Organisation for 
Standardisation, ISO MPEG Document, ISO-IEC/JTC1/SC2/WG1 1, 1994), 
and the emerging standard MPEG4, which includes further added 
5 functionality and quality over MPEG1 and MPEG2. 
[0003] 

In MPEG1 and MPEG2 decoders, there are certain expectations 
regarding the decoder computational power which must be met in order to 
conform to the respective standard. For example, in the MPEG2 standard, 
10 Main Profile and Main Level (MP@ML) describes a set of bounds on the 

bitstream and associated video which all conformant MP@ML decoders must 
adhere to. 
[0004] ! 

There are many situations where a decoder lacks sufficient 
15 computational power to decode a bitstream as it was intended. One 

example is on the internet where typically MPEG1 video bitstreams can be 
found and decoded using computers varying from high end scientific work 
stations to simple personal computers. When computational power is 
limited, one way the decoder can reduce it's computational requirements is to 
20 simply slow down the decoder output picture rate enough to permit all 

decoding to be accomplished. This was a very common method used by 
software based MPEG1 video decoders running on personal computers. 
Another method is to periodically skip the decoding of some pictures. For 
example, bi-directionally predicted pictures (B-pictures) can be skipped, 
25 effectively reducing the decoder output picture rate by picture-dropping. 
[0005] 

MPEG4 includes the concept of video objects, which are picture 
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sequences which may be used to describe a separable object in a scene. If 
a scene contains many objects, the least important objects can be dropped in 
favour of giving the necessary computational power to the decoding of the 
more important objects. 
5 [0006] 

[Problem to be solved by the invention] 

There is a growing abundance of digital video bitstreams available, 
and a growing number of computational requirements associated with each. 
Due to the wide range of platforms that can be used for video decoding, 
10 there is a need for decoders which can decode bitstreams without requiring 
the full computational power required to meet conformance criteria. 
-* Existing techniques such as picture display rate reduction, picture dropping 

^ or object dropping techniques reduce the computational power at the 

expense of decoded video picture quality and synchronisation. For example, 
is changing the picture display rate and picture dropping can have an effect on 
the perceived synchronisation of audio and video, commonly referred to as 
lip-synch. It also has the effect of making scenes with motion appear jerky. 
[0007] 

An object of this invention is to permit video decoders to reduce the 
2 0 computational requirements of decoding a bitstream without affecting the 

picture rate. This will permit lip-synch to be maintained while compromising 
visual quality. It will also permit scenes which become jerky using picture 
dropping techniques to appear more smooth. Similarly, it will permit more 
objects to be present in the scene, since the computational power can be 
25 allocated to more objects, at the expense of object quality. 
[0008] 

An object of this invention is to permit video decoders which lack the 
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computational power to decode a bitstream to conformance, to decode a 
bitstream with varying degrees of visual quality degradation. This will 
permit a wide variety of decoders to be made, each with varying 
cost/performance trade-offs. An expensive decoder with a very high 
computational power can be used to decode bitstreams with a much better 
resulting quality than an inexpensive decoder with low computational power. 

By using this invention, the same bitstream may be decoded by a wide 
range of computational power decoders, to result in varying degrees of 
acceptable visual quality. 
[0009] 

An object of this invention is to reduce the cost of video decoders. By 
designing video decoders that are able to meet constraints such as ; those 
imposed by MPEG2 MP@ML limitations, excessive computational power 
overhead must be built into the decoder to guarantee real time picture 
decoding of worst-case (high complexity) bitstreams. In other words, the 
pictures are decoded on time even when the worst case bitstream is decoded. 

However, by using this invention, decoders can be made which gracefully 
degrade picture quality when their performance limitations are met. 
[0010] 

[Means for solving the problem] 

For the purpose of solving the above described problems, the method 
of reducing processing power requirements of a video decoder described 
herein was invented. Said processing power requirements may be 
controlled based on a throttling amount. Said throttling amount may be 
based on one or more measures of the processing power required to decode 
one or more bitstreams, one or more measures of said decoder's processing 
capabilities, or some combination of these measures. 
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[0011] 

The measures of the processing power required to decode a bitstream 
may comprise of an indication of the syntax elements in said bitstream, an 
indication of the type of processing required for said bitstream, an indication 
5 of the amount of processing required for said bitstream, or some combination 
of these indications. Reduction of processing power requirements is 
accomplished by reducing the processing power used for one or more 
decoder functions by limiting said decoder function in some way. Said 
limiting may comprise turning off the use of one or more of the motion vectors 
10 normally used to retrieve a motion compensated reference. This may be 

done by selecting from a choice of motion vectors which normally are used to 
retrieve a pixel regions motion compensated reference predictor region, one 
ha**>** or more motion vectors whose use is turned off. Selection of said motion 

vectors may comprise of selecting the motion vectors which are of temporally 
**** 15 longest distance, selecting the motion vectors which are least highly 

correlated with those motion vectors in surrounding regions of the picture 
region being decoded, selecting the motion vectors which are of spatially 
longest distance, selecting all motion vectors, or any combination of these 
and other selection criteria. 
20 [0012] 

Another method for limiting said decoder function comprises of 
modifying motion vectors, possibly by limiting the precision and/or range of 
one or more of the decoder's motion vectors. Another method for limiting 
said decoder function comprises of reducing the number of coefficients 
25 inverse quantized and inverse DCT transformed by selectively setting 

coefficients to alternate values, such as zero. Another method for limiting 
said decoder function comprises of reducing the amount of decoder 
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processing used for decoding the color components of the video bitstream. 
Another method for limiting said decoder function comprises of reducing the 
amount of processing performed on the decoder's output video pictures. 
[0013] 

5 Said throttling amount serves to control which and by how much to 

limit said decoder functions in some way. It is possible to determine the 
processing power requirements from one or more bitstreams, as well as the 
existing processing power capabilities of the decoder, and use this 
information to make a decision about which and how much to throttle, or limit 

10 said decoder functions during the decoding of said bitstreams By limiting 
the decoder functionality, the processing power can be reduced. The 
identification of which functions to limit and in what way can be made using 
^indications such as the syntax elements in said bitstream, and the type and 
amount of processing required for said bitstream. 

15- [00014]* 

Said limiting may comprise turning off the use of one or more of the 
motion vectors normally used to retrieve a motion compensated reference, 
based on a decision made with reference to indications in the bitstream about 

J 

what motion vectors exist, or some other indications from the bitstream or 
20 decoder. The choice of what motion vectors in a picture to turn off can be 
further refined by selecting the motion vectors which are of temporally 
longest distance, selecting the motion vectors which are least highly 
correlated with those motion vectors in surrounding regions of the picture 
region being decoded, selecting the motion vectors which are of spatially 
25 longest distance, selecting all motion vectors, or any combination of these 
and other selection criteria. 
[0015] 
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Similarily, said limiting may comprise of limiting the precision and/or 
range or some other modification of one or more of the decoder's motion 
vectors. Another method for limiting said decoder function comprises of 
reducing the number of coefficients inverse quantized and inverse DCT 
5 transformed by selectively setting coefficients to alternate values which are 
easy to inverse quantize and inverse DCT, such as zero. The decision to 
limit these decoder functions may be based on information regarding said 
bitstream and/or said decoder. Another method for limiting said decoder 
function comprises of eliminating some or all of the processing required 

10 during color component decoding. Another method for limiting said decoder 
function comprises of limiting the processing applied to the decoded video 
pictures. 

- [0016] 

[Embodiments of the Invention] 

15* ■* An example embodiment of this invention is shown in Figure 1. One 

or more bitstreams are input via the bitstream input (100, 101). The video 
decoder (110) decodes the bitstream and outputs decoded video (120, 121). 

Measures of the bitstreams' processing requirements (130) and decoder's 
processing capabilities (131) are passed from the video decoder (110) to the 

20 decoder throttler (140). Throttling amount (132) is passed from the decoder 
throttler (140) to the video decoder (1 10). 
[0017] 

The operation of the embodiment in Figure 1 is now described. The 
video decoder (110) extracts measures of the bitstream processing 
25 requirements from the input bitstreams (101) and passes these measures 
(130) to the decoder throttler (140). The video decoder (110) also 
determines it's capabilities for decoding the bitstream and passes this 
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measure (131) to the decoder throttler (140). The decoder throttler (140) 
uses these measures (130, 131) to decide how the video decoder (110) 
function can be throttled and passes this throttling amount (132) information 
to the video decoder (110). 
[0018] 

It is possible that no measures of the bitstreams (130) are taken and 
passed to the decoder throttler (140). In this case, the decoder throttler 
(140) determines how to throttle the video decoder (110) by using the 
measure of the decoder's processing capabilities (131), or some other 
measure. 
[0019] 

It is also possible that no measures of the decoder's processing 
capabilities (131) are taken and passed to the decoder throttler (140). In 
this case, the decoder throttler (140) determines how to throttle the video 
decoder (110) by using the measures of the bitstream processing 
requirements (130), or some other measure. 
[0020] 

It is possible that no measures of the bitstreams (130) or decoder's 
processing capabilities are taken and passed to the decoder throttler (140). 

In this case, the decoder throttler (140) determines how to throttle the video 
decoder (110) by using some other measure, or by using some 
predetermined method. For example, it is possible that due to some 
information known about the decoder(1 10) and decoder throttler (140) when 
their architectures are designed, the decoder throttler (140) can always 
throttle in the same way. The number of measures and origin of the 
measures is not limited by this invention. 
[0021] 
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The effect of the embodiment described by Figure 1 is the ability of the 
video decoder to throttle it's performance requirements. 
[0022] 

Another example embodiment of this invention is shown in Figure 2. 
One or more bitstreams are input via the bitstream input (200, 201 ). The 
video decoder (210) decodes the bitstream and outputs decoded video (220, 
221). The input bitstreams (202) are passed to the bitstream indicator 
extractor (250) where measurements are made concerning the bitstream. 
Indications such as syntax elements (251), type of processing required for 
decoding (252), and the amount of processing required (253), are passed to 
the decoder throttler (240). The decoder's processing capabilities (231) are 
passed from the video decoder (210) to the decoder throttler (240). 
Throttling amount (232) is passed from the decoder throttler (240) to the 
video decoder (21 0). 
[0023] 

The operation of the embodiment in Figure 2 is now described. The 
bitstream indicator extractor (250) extracts measures of the bitstream 
processing requirements from the input bitstreams (202). It determines 
indicators which characterise the stream's contents and types and amounts 
of processing required to decode the bitstreams and passes these indicators 
(251 , 252, 253 respectively) to the decoder throttler (240). The video 
decoder (210) also determines it's capabilities for decoding the bitstream and 
passes this measure (231) to the decoder throttler (240). The decoder 
throttler (240) uses these indicators (230, 231) to decide how the video 
decoder (210) function can be throttled and passes this throttling amount 
(232) information to the video decoder (210). 
[0024] 
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It is possible that some or all of the indications (251 , 252, 253) taken 
by the bitstream indicator extractor (250) are not used by the decoder 
throttler (240). In this case, the decoder throttler (240) determines how to 
throttle the video decoder (210) by using the measure of the decoder's 
5 processing capabilities (231), or some other measure. 
[0025] 

It is also possible that no measures of the decoder's processing 
capabilities (231 ) are taken and passed to the decoder throttler (240). In 
this case, the decoder throttler (240) determines how to throttle the video 
10 decoder (210) by using the indications (251, 252, 253) determined by the 
bitstream indicator extractor (250), or some other measure. 
[0026] 

- ' It is possible that no bitstream indications (251, 252, 253) or decoder's 

processing capabilities (231) are taken and passed to the decoder throttler 

15 (240). In this case, the decoder throttler (240) determines how to throttle 
the video decoder (210) by using some other information, or by using some 
predetermined method. For example, it is possible that due to some 
information known about the decoder(210) and decoder throttler (240) when 
their architectures are designed, the decoder throttler (240) can always 

20 throttle in the same way. The number of indicators and measures and the 
origin of the indicators and measures is not limited by this invention. 
[0027] 

Figure 2 and Figure 3 can be used to describe another embodiment of 
this invention. In this embodiment, an MPEG2 video decoder is used. The 
25 following are extracted by the bitstream indicator extractor (250): 
[0028] 

□ picture_coding_type (I, P, B,D) 
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□ temporal_reference 

Figure 3 describes a flow chart used to determine which motion 
vectors (forward or backward) in a B-picture are to be discarded. The 
temporaMreference information is used to determine whether the forward 
or backward reference is of further temporal distance from the picture being 
decoded. The reference of furthest distance is not used, effectively turning 
off motion vectors referring to it. In other words, the B-picture only uses 
uni-directional prediction in this embodiment. 
[0029] 

The picture_coding_type and temporal_reference are not the only 
bitstream characteristics which can be extracted and used for throttling the 
video decoder (210). This invention does not limit the type or number of 
indicators used for determining the way to throttle the video decoder (210). 

The invention does not limit the characteristics to specific syntax elements. 

For example, the characteristics could be indicators that are not specific 
syntactic elements, like indicators of the number of macroblocks of a 
particular type. The indicators may have been inserted into the bitstream by 
the encoder, or may be implied by some bitstream elements. The indicators 
may be at a higher syntactic level, for example, the system bitstream level. 

The indicators taken from the bitstream for a particular element may not be 
used to control the decoding of that particular element. For example, the 
indicators associated with one picture may be used to control the decoding of 
another picture. For example, the indicators of a preceding picture could be 
used to control the decoding of the current picture. 
[0030] 

The invention is also not limited to the frequency of video decoder 
(210) throttling as described by the embodiment described in Figure 3. For 
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example, the decision of whether to use bi-directional prediction may be 
made on a pixel region by region case. For example, the decision could be 
re-evaluated every macroblock. 
[0031] 

5 Also, the previous embodiment did not describe the use of measures 

of the decoder processing capability (231). These can be used to detect 
when a decoder is very busy, at which time it cannot achieve real time 
decoding of the bitstreams, and so decoder throttling is needed. Likewise, 
the measures of decoder processing capability (231 ) can indicate when the 
10 decoder is not busy, and can achieve real time, video decoding with no 

throttling of it's functions. In other words, the method of making the decision 
of whether or not to turn off some motion vectors and their associated 

reference compensation processing, is not limited by this invention. 

[0032] 

is ***** The effect of this embodiment is the significant reduction of frame 
memory accessing and processing for B-pictures. Reference frame 
memory reading is effectively cut in half due to the absence of the bi- 
directional processing of motion compensated references. Although the 
quality of the resulting decoded video is degraded, this is considered an 

20 acceptable side-effect of the invention, since the invention is used to reduce 
decoder cost by degrading the picture quality. Since B-pictures are not 
used for further predictions, the errors in the pictures introduced by the 
throttling do not propagate to other pictures. 
[0033] 

25 Figure 4 shows an example embodiment in which the picture size and 

rate is used as an indication of the amount of processing (253) which is 
required by a bitstream. In this embodiment, the following additional 
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elements are determined from a bitstream: 
[0034] 

□ horizontal_size 

□ vertical_size 
> □ frame_rate_value 

These are then used to decide if the decoder will employ some 
throttling techniques or not, based on the bitstreams required decoder output 
pixel rate In the example shown, it is used, but is not limited to, enable the 
motion vector throttling described by Figure 3. 
10 [0035] 

In Fig. 3, |TR - TR for | is the absolute value of the difference in time of 
— the picture being decoded and the picture used as the motion compensated 
1 ^# - forward reference, and |TR - TR back | is the absolute value of the difference in 

time of the picture being decoded and the picture used as the motion 
15 compensated backward reference. 
[0036] 

Another embodiment of the invented method modifies the motion 
vectors in a way that reduces the computational and/or memory bandwidth 

'i 
i 

requirements of the decoder. 
2 0 [0037] 

One embodiment utilises the precision of motion vectors. Figure 5 
describes how 14 pixel motion compensated references are determined by 
using the neighbouring full pixel values. If a spatial region required !4 pixel 
interpolations, more reference frame memory pixel data is needed. For 
25 example, if an 8x8 pixel region is desired using full pixel motion vectors, an 
8x8 region would be required to be read. On the other hand, if both 
horizontal and vertical 1 /4 pixel motion compensation was required, a 9x9 
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region must be read. After V* pixel interpolations of nearest neighbour full 
pixel values, the desired 8x8 region would be determined. Thus, it is 
possible to reduce a video decoder's processing requirements by limiting the 
precision of it's motion vectors to full pixel precision. 
5 [0038] 

This invention does not limit the reducing of the precision of the motion 
vectors from V£ pixel to full pixel values. In fact, many video decoders do not 
access the reference frame memories pixel by pixel, but instead group the 
pixels into accessible words. Thus, limiting motion vector precision to the 
10 word size of the reference frame memories can be performed. 
[0039] 

The effect of this embodiment is the reduction of frame memory 
— - accessing and processing for pictures using motion compensated prediction. 

Although the quality of the resulting decoded video is degraded, this is 
^ # 15 considered an acceptable side-effect of the invention, since the invention is 
used to reduce decoder cost by degrading the picture quality. 
[0040] 

Another embodiment of the invented method reduces the motion 

J 

vector range. For example, if the range of motion vectors is +/- 64 pixels, 
20 the range can be limited to +/- 16 in the decoder. The limited range is not 
specific to this embodiment. For example, it could be limited to 0 pixels. 
[0041] 

The effect of this embodiment is that the frame memory accessing is 
restricted to a smaller region of the picture. This can be particularly useful 
25 in motion compensation architectures which use memory cache. The more 
pixels which are accessed from cache, the fewer cache hits. Cache hits 
mean that the desired pixel is not in the cache memory, so they must be 
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fetched from a typically bigger and slower memory which holds the desired 
pixels. The overall effect of a cache hit is a reduced processing 
performance. The effect of reducing the motion vector range is to reduce 
the number of cache hits, and thus improve processing performance. 
[0042] 

Another embodiment of the invented method modifies the values of 
some of the coefficients decoded from the bitstream in order to make the 
downstream processing of these coefficients less computationally intensive. 
An example of such a modification is shown in Figure 6. The original 
coefficients from the bitstream (600) are modified to result in the modified 
values (601). In this example, all values outside of the grey area (602) are 
set to zero. The effect of this in a DCT based system such as MPEG2 for 
example, where a video decoder typically performs inverse quantization 
before inverse DCT, is that by setting the coefficient values decoded from the 
bitstream to values easily inverse quantized, the processing requirements for 
inverse quantization can be reduced. For example, if some of the 
coefficients are set to zero, the result after inverse quantization of these 
coefficients can be set to zero without the usual calculations as defined by 
the MPEG2 standard. In addition, some implementations of an inverse DCT 
utilize detected coefficients that are zero to reduce the computational 
requirements of the inverse DCT. For example, in a separable inverse DCT 
in which a subset of the coefficients are to be transformed, if the subset is all 
zero, the results will also be all zero. In this case the inverse transform can 
be completed much easier simply by setting the results to zero rather than by 
calculating the inverse transform. 
[0043] 

This invention is not limited to setting coefficient values from the 
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bitstream to zero. They can be modified to any value which results in less 
downstream processing. It is also does not limit which values are modified. 

For example, it is possible to limit the number of non-zero coefficients in a 
block simply by stopping the saving of the values from the bitstream once the 
5 limit for the block is reached. This invention is not limited to decoders using 
the inverse DCT as it's transform. Other transform's coefficients can also 
be modified to reduce their computational requirements. Another 
embodiment of this invention is the reduction or elimination of the processing 
of one or both of the color components. For example, the coefficient values 
10 can be discarded and the pixel values for the color components set to neutral. 

In this case, there is no further processing of the color components required 
— ^ by the decoder. For example, inverse quantization, inverse DCT and 

... ^m*^,^,. -motion compensation are not required for the color components. The effect 
■■■ of this embodiment is the significant reduction of decoder processing 
15 requirements. Another effect is the loss of color in the displayed picture, but 
this may be an acceptable degradation due to the reduced decoder 
computations. 
[0044] 

Another embodiment of this invention is the reduction of processing 
20 performed on the decoded video pictures. For example, post filters can be 
used by a video decoder to remove some blocky noise introduced by the 
encoding and decoding processes. The effect of reducing this filtering is the 
reduction of the computational requirements of the post filter. The reduction 
of the processing can be the reduction of the number of taps used by the post 
25 filter. Other forms of post processing can also be reduced, such as format 
conversion filters. 
[0045] 
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There are many combinations of the invented methods which can be 
used to reduce the processing power requirements of a video decoder. This 
invention does not limit the combinations. For example, motion vectors can 
be modified, the coefficients extracted from the bitstream can by modified, 
5 and the processing performed on the decoded pictures limited within the 
same decoder. 
[0046] 

The overall effect of this invention is the great freedom of control over 
the processing requirements and output picture quality of a video decoder. 
10 This control is useful when decoding single bitstreams or multiple 
bitstreams that may have associated priorities. 
[0047] 

— [Effect of the invention] 

One effect of this invention is that the processing power of video 
15 decoders can be reduced, permitting bitstreams with high computational 
requirements to be decoded by decoders with lower computational ability. 
By basing the decision of the type and amount of decoder function limiting on 
measures of the processing power required to decode bitstreams, measures 

) 

of said decoder's processing capabilities, or some combination of these 
20 measures, an appropriate choice of limiting can be made which balances the 
trade-off between computational requirements and picture quality. An effect 
of using indications of the syntax elements in a bitstream and types and 
amounts of processing in said bitstream is that the type and amount of 
decoder function limiting can be made considering the content of the 
25 bitstream, thus permitting a more suitable and visually appealing choice for 
function limiting. 
[0048] 
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An effect of using measures of the decoder's processing capabilities is 
that it permits the selection of functions to limit to be based on the decoder's 
abilities, which is dependant on the decoder's architecture. An effect of 
reducing the computational power of one or more functions in a decoder is 
the reduction of the decoder's computational power. An effect of turning off 
the use of one or more of the motion vectors normally used to retrieve a 
motion compensated reference is a reduction in processing requirements 
and reference frame memory access bandwidth. 
[0049] 

An effect of selectively turning off the use of one or more motion 
vectors is that it is possible to select those motion vectors to turn off which 
have a less detrimental affect on picture quality than others, thus permitting a 
trade-off between picture quality and computational requirements. An effect 
of limiting some of the motion vectors which are of temporally longest 
distance is that the visual quality is typically less affected by turning off some 
of these than it is for motion vectors of shorter temporal distance. An effect 
of limiting the motion vectors which are least highly correlated with those 
motion vectors in surrounding regions of the picture region being decoded is 
that the perceptibility of degrading the picture quality of the regions with 
these motion vectors may be less than that of degrading more highly 
correlated motion regions. 
[0050] 

It also has the effect of permitting cache based architectures for 
motion compensation functions to have fewer cache hits if the less highly 
correlated motion vectors are not used. An effect of selecting the motion 
vectors which are of spatially longest distance is that it may permit cache 
based architectures to have fewer cache hits if the motion references are 
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more close together, since presumably the reference data would not be going 
outside of the cache so often. An effect of modifying the motion vectors or 
limiting the precision and/or range of one or more of the decoders motion 
vectors is that the number of memory accesses can be reduced when 
accessing motion compensated references. 
[0051] 

An effect of reducing the number of coefficients inverse quantized and 
inverse DCT transformed by selectively setting coefficients to alternate 
values, such as zero, is that the computational requirements of inverse 
quantization and inverse DCT can be reduced. An effect of reducing the 
processing of the color components is the overall reduction of decoder 
processing. 

[Brief description of drawings] 

Figure 1 : Block diagram of first embodiment of the invention 

Figure 2 : Block diagram of second embodiment of the invention 

Figure 3 : A flow chart of the embodiment using motion vector throttling 

Figure 4 : A flow chart of the embodiment using picture size and rate 

Figure 5 : A diagram showing V£ pixel motion compensation 

Figure 6 : A diagram showing modifying coefficient values 

[Description of symbols] 

100,101, ---bitstream input 

110 • -video decoder 

120,121 - decoded video 

140 -decoder limiter (throttler) 

200, 201 -bitstream 

210 - video decoder 

240 - decoder limiter (throttler) 



-24- 




JPA 9-283640 



250 •bitstream indicator extractor 
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[Document name] Abstract 

[Abstract] 

[Problem] 

The object is to present video decoders of varying degrees of 
5 computational power to decode bitstreams with varying degrees of visual 
quality degradation. 
[Means for solving problem] 

To match with the video decoder's maximum processing power, the 
processing power requirements of a video decoder based on both bitstream 
10 contents and decoder capabilities are reduced. One method it uses is the 
reduction of motion compensation processing by modifying the use or values 
of motion vectors, such as by turning some motion compensation off, or by 
limiting the precision of motion vectors during decoding. Another method is 
by limiting the coefficient processing so as to reduce the computational 
15 requirements. Another method is to limit the processing of color 

components, and another method is by reducing the amount of filtering 
performed on the decoder's output video pictures. 

[Selected figure] Fig. 1 
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